Skip to content

FSST: decompress straight into the builder's byte storage - #9097

Open
robert3005 wants to merge 1 commit into
rk/varbin-builder-traitfrom
rk/fsst-decode-into-builder
Open

FSST: decompress straight into the builder's byte storage#9097
robert3005 wants to merge 1 commit into
rk/varbin-builder-traitfrom
rk/fsst-decode-into-builder

Conversation

@robert3005

@robert3005 robert3005 commented Jul 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

FSST append_to_builder decodes directly into the builder storage without additional copy

@robert3005 robert3005 added the changelog/chore A trivial change label Jul 31, 2026
@robert3005
robert3005 marked this pull request as ready for review July 31, 2026 13:40
@robert3005
FSST: decompress straight into the builder's byte storage
96d2f3d 
`append_to_builder` staged the decompressed heap in a temporary
`ByteBufferMut` and then copied it into the builder. `FsstDecodePlan` hoists
the parts of `fsst_decode_bytes` that describe the decode — the code stream,
the per-row lengths and the total decoded size — so the decompression can
target the builder's spare capacity directly via `append_decoded`, leaving one
prefix sum over the lengths as the only work beyond the bulk
`decompress_into`.

`FSST_DECODE_SLACK` names the 7 spare bytes that keep `decompress_into` on its
wide-store path through the final value; it is a performance knob, not a safety
requirement.

Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
Signed-off-by: Robert Kruszewski <github@robertk.io>
@robert3005
robert3005 force-pushed the rk/fsst-decode-into-builder branch from 2de8a74 to 96d2f3d Compare July 31, 2026 18:11
@codspeed-hq

codspeed-hq Bot commented Jul 31, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 94.77%

⚡ 6 improved benchmarks
✅ 1836 untouched benchmarks
⏩ 55 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ Simulation extend_from_array_non_zctl_overlapping[(10000, 8)] 2,461.2 µs 707.9 µs ×3.5
⚡ Simulation extend_from_array_non_zctl_overlapping[(1000, 32)] 892.2 µs 292.8 µs ×3
⚡ Simulation extend_from_array_zctl[(10000, 8)] 2.2 ms 1.2 ms +83.8%
⚡ Simulation extend_from_array_zctl[(1000, 64)] 1,200.8 µs 730.8 µs +64.3%
⚡ Simulation extend_from_array_non_zctl_overlapping[(1000, 8)] 428.9 µs 296.3 µs +44.77%
⚡ Simulation extend_from_array_zctl[(1000, 8)] 401.6 µs 340.7 µs +17.87%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

Comparing rk/fsst-decode-into-builder (96d2f3d) with rk/varbin-builder-trait (b81a551)2

Open in CodSpeed

Footnotes

  1. 55 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on rk/varbin-builder-trait (e072374) during the generation of this report, so 153f08d was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

changelog/chore A trivial change

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@robert3005